Reverse correlation demo
Computational Vision
University of Minnesota
Psy 5036

Demonstration of yes/no classification images

Initialization

Off[General::"spell1"];
SetOptions[ArrayPlot,ColorFunction->"GrayTones",DataReversed->True,Frame->False,AspectRatio->Automatic,Mesh->False,PixelConstrained->True,ImageSize->Small];

Introduction

Imagine you have a mental picture in mind, say of the letter B that you want to convey to someone. You could try to describe it, or perhaps better, draw it. Another way would be to look at a series of images and pick one out that is close. But better, suppose you could look at a large series of pictures and pick out a subset that is close, and then somehow average those to produce an image that corresponds to what you are imagining. There exists a technique like this, called "reverse correlation analysis" or "response classification, to produce  "classification images". The idea in its simplest form, is to have an observer such as yourself, look at a large series of white noise pictures, and respond to which ones look like the pattern you have in mind. Mark those as "positive" and the others as "negative". Average the images with positive responses and separately average those with negative responses. Show the images, and in particular the difference between the the positive and negative provide a representation of a "classification image". Most applications of this idea have been more refined and address other issues, like the representation of illusory contours (Gold et al., 2000)  orientation tuning in human vision (Ringach, 1998). The theory has also been  developed and expanded (e.g. special issue of the Journal of Vision, Eckstein & Ahumada, editors, 2002).

We illustrate the idea below, where we use the computer to hold a pattern "in mind".

Simulation

template of the pattern in mind -- letterB "-- to be discovered".

letterB

template=Map[0.3333#[[1]]+0.333#[[2]]+0.333#[[3]]&,N[letterB],{2}];
Graphics[Raster[letterB]];
ArrayPlot[template];

Specify image size, and number of trials to run

width=Dimensions[template][[1]];
size=width*width;
numtrials = 5000;

Define noise function

sigma=.375; (*noise level*)
ndist=NormalDistribution[0,sigma];
noise:=Table[Random[ndist],{size}];

Initialize positive and negative image buffers

CP=Table[0.0,{size}]; (*Accumlates noise images that are similar to template*)
CN=Table[0.0,{size}];(*Accumulates noise images that are not similar to template*)

Run a series of trials. If the random noise sample is "similar to the template", add it to CP, othewise add the noise to CN.
"Similarity" is measured by whether its dot product with the template is bigger than zero or not. You could try other measures.

i=0;
ftemplate=Flatten[template];
Timing[While[i<numtrials,
noisesample=noise;
If[noisesample.ftemplate>0,CP=CP+noisesample,CN=CN+noisesample];
i++];]

Calculate and display the classification image, Cimage

Cimage=Partition[CP-CN,width];

ArrayPlot[Cimage,Mesh->False]

Try it with a grating template

width = 16; size = width*width;

cgabor[x_,y_, fx_, fy_,s_] := 
Exp[-(x^2 +  y^2)/s^2] Cos[2 Pi(fx x + fy y)];

vtheta=Table[i1 Pi/4,{i1,4}]; (*various orientations to try*)
vf={1,1,2,4};  (*various spatial frequencies to try*)
swidth={.25,1,4}; (*various widths*)
(*Pick one set of parameters*)
template = Table[0.0625*N[cgabor[x,y,vf[[1]] Cos[vtheta[[1]]],vf[[1]] Sin[vtheta[[1]]],swidth[[2]]]],{x,-2,2,4/(width-1)},{y,-2,2,4/(width-1)}];

ArrayPlot[template,Mesh->False]

References

Ahumada, A.J., Jr.(2002).Classification image weights and internal noise level estimation.J Vis, 2 (1), 121 - 131.

Eckstein, M. P., & Ahumada, A. J., Jr. (2002). Classification images: A tool to analyze visual strategies. Journal of Vision, 2(1):i, i, http://journalofvision.org/2/1/i/, doi:10.1167/2.1.i.

Gold, J. M., Murray, R. F., Bennett, P. J., & Sekuler, A. B. (2000). Deriving behavioural receptive fields for visually completed contours. Curr Biol, 10(11), 663-666.

Olman, C. A., & Kersten, D. (2004). Classification objects, ideal observers & generative models. Cognitive Science, 28, 227-239.

Ringach, D. L. (1998). Tuning of orientation detectors in human vision. Vision Res, 38(7), 963-972.